[SPARK-17525][Python] Remove SparkContext.clearFiles() from the PySpark API as it was removed from the Scala API prior to Spark 2.0.0 #15081
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rk API as it was removed from the Scala API prior to Spark 2.0.0
Contributor
Author
|
cc @rxin @JoshRosen since you were around when the deprecated APIs were removed. |
Member
|
@sjakthol it seems the JIRA in the title is wrongly linked the title. Could you confirm this please? |
Contributor
Author
|
I can confirm. It's SPARK-17525 instead of SPARK-17252 :) Updating. |
sjakthol
changed the title
Contributor
|
Yep, good catch but I think the JIRA number is wrong; could you change it to SPARK-17525? |
Contributor
|
Doh, I see that it's updated already (my page was out of date). LGTM pending Jenkins. |
|
Test build #3261 has finished for PR 15081 at commit
|
Member
|
Merged to master/2.0 |
asfgit
pushed a commit
that referenced
this pull request
Sep 14, 2016
…rk API as it was removed from the Scala API prior to Spark 2.0.0 ## What changes were proposed in this pull request? This pull request removes the SparkContext.clearFiles() method from the PySpark API as the method was removed from the Scala API in 8ce645d. Using that method in PySpark leads to an exception as PySpark tries to call the non-existent method on the JVM side. ## How was this patch tested? Existing tests (though none of them tested this particular method). Author: Sami Jaktholm <[email protected]> Closes #15081 from sjakthol/pyspark-sc-clearfiles. (cherry picked from commit b5bfcdd) Signed-off-by: Sean Owen <[email protected]>
wgtmac
pushed a commit
to wgtmac/spark
that referenced
this pull request
Sep 19, 2016
…rk API as it was removed from the Scala API prior to Spark 2.0.0 ## What changes were proposed in this pull request? This pull request removes the SparkContext.clearFiles() method from the PySpark API as the method was removed from the Scala API in 8ce645d. Using that method in PySpark leads to an exception as PySpark tries to call the non-existent method on the JVM side. ## How was this patch tested? Existing tests (though none of them tested this particular method). Author: Sami Jaktholm <[email protected]> Closes apache#15081 from sjakthol/pyspark-sc-clearfiles.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This pull request removes the SparkContext.clearFiles() method from the PySpark API as the method was removed from the Scala API in 8ce645d. Using that method in PySpark leads to an exception as PySpark tries to call the non-existent method on the JVM side.
How was this patch tested?
Existing tests (though none of them tested this particular method).